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CLAIMS 



What is Claimed is: 



1 . A method for optimizing an estimate of a depth map of a reference image 
through the blending of a plurality of depth maps, taken two depth maps at a time, 
comprising: 

calculating the reprojection energies of assigning each of two adjacent pixels 
of a reference image to each of two separate depth maps; 

calculating the discontinuity energies associated with each pixel of the 
adjacent pixels of the reference image and associated with the edge between the adjacent 
pixels of the reference image; and 

assigning depth map values for the two adjacent pixels based on a minimum 
graph cut between the two separate depth maps, given the adjacent pixels and the calculated 
reprojection and discontinuity energies. 

2. The method according to claim 1, wherein the step of assigning depth map 
values further includes: 

adjusting the calculated reprojection energies with the calculated discontinuity 

energies; 

determining the energy costs associated with assigning the two separate depth 
maps to the adjacent pixels; and 

assigning depth map values for the two adjacent pixels based on the minimum 
energy cost associated with assigning the two separate depth maps to the adjacent pixels. 

3. The method according to claim 1, wherein the two separate depth maps consist 
of a first, estimated depth map and a second, hypothetical depth map and wherein the step of 
assigning depth map values includes replacing depth map values of the first, estimated depth 
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map to produce a third, optimized depth map which then becomes the first, estimated depth 
map for subsequent optimization iterations. 

4. The method according to claim 3, wherein the second, hypothetical depth map 
is a complex, non-planar depth map. 

5. The method according to claim 3, wherein the two adjacent pixels constitute a 
neighboring pixel pair. 

6. The method according to claim 5, further including repeating the steps of 
calculating the reprojection energies, calculating the discontinuity energies, and assigning 
depth map values for each pixel pair of the reference image until the difference between the 
depth map values assigned at each iteration of the reference image pixel pair set reaches a 
predetermined minimum. 

7. The method according to claim 6, further including deriving a new second, 
hypothetical depth map for further processing when the difference between the depth map 
values assigned at each iteration of a reference image pixel pair set reaches a predetermined 
minimum. 

8. A method for estimating a depth map of a reference image through the 
blending of a plurality of depth maps, taken two depth maps at a time, comprising: 

estimating a current depth map of a specific view of a reference image; and 
for each of a plurality of derived hypothetical depth maps of the reference 
image, performing the following: 

for each pixel on the current depth map that corresponds to a pixel on 
the hypothetical depth map, comparing the depth map value of the pixel on the 
current depth map with the depth map value of the pixel on the hypothetical 
depth map; and 
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replacing the depth map value of the pixel on the current depth 
map with the corresponding depth map value of the pixel on the 
hypothetical depth map if the compared depth map value of the pixel 
on the hypothetical depth map has a higher probability of accurately 
representing the reference image than does the compared depth map 
value of the pixel on the current depth map. 

9. The method according to claim 8, wherein the view each of the plurality of 
hypothetical depth maps includes at least a subregion of the view of the current depth map. 

10. The method according to claim 8, wherein one or more of the plurality of 
hypothetical depth maps is a complex, non-planar depth map. 

1 1 . The method according to claim 8, wherein the comparing of depth map values 
is terminated once the difference between the depth map values of the current depth map and 
the depth map values of the derived hypothetical depth map reaches a predetermined 
minimum. 

12. The method according to claim 8, wherein the comparing of depth map values 
is performed a plurality of times across all pixels of the reference image until the difference 
between the depth map values of the current depth map and the depth map values of the 
derived hypothetical depth map reaches a predetermined minimum. 

13. The method according to claim 8, wherein the probability of accurately 
representing the reference image is determined according to a Bayesian framework. 

14. The method according to claim 13, wherein the probability of accurately 
representing the reference image is determined according to energy costs and graph cuts. 
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15. A method for optimizing an estimate for a depth map of a reference image of 
an object, comprising: 

estimating a first depth map of a desired view of a reference image of an 

object; and 

for each of a plurality of derived hypothetical depth maps of the reference 
image, performing the following: 

for every pixel within both the first depth map and the derived 
hypothetical depth map, applying a Bayesian probability framework to determine the 
optimum depth map value between the two depth maps, wherein said determination is 
accomplished by minimizing the energy costs associated with graph cuts between 
neighboring pixel pairs; and 

replacing the depth map value in the first depth map with the 

optimum depth map value. 

16. A system for optimizing an estimate of a depth map of a reference image 
through the blending of a plurality of depth maps, taken two depth maps at a time, 
comprising: 

a first processor calculating the reprojection energies of assigning each of two 
adjacent pixels of a reference image to each of two separate depth maps; 

a second processor calculating the discontinuity energies associated with each 
pixel of the adjacent pixels of the reference image and associated with the edge between the 
adjacent pixels of the reference image; and 

a third processor assigning depth map values for the two adjacent pixels based 
on a minimum graph cut between the two separate depth maps, given the adjacent pixels and 
the calculated reprojection and discontinuity energies. 

17. The system according to claim 16, wherein the third processor further 
includes: 
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a fourth processor adjusting the calculated reprojection energies with the 
calculated discontinuity energies; 

a fifth processor determining the energy costs associated with assigning the 
two separate depth maps to the adjacent pixels; and 

a replacement device assigning depth map values for the two adjacent pixels 
based on the minimum energy cost associated with assigning the two separate depth maps to 
the adjacent pixels. 

18. A system for estimating a depth map of a reference image through the 
blending of a plurality of depth maps, taken two depth maps at a time, comprising: 

a first processor estimating a current depth map of a specific view of a 
reference image; and 

a second processor comprising the following for each of a plurality of derived 
hypothetical depth maps of the reference image: 

a third processor comprising the following for each pixel on the current 
depth map that corresponds to a pixel on the hypothetical depth map: 

a comparison device comparing the depth map value of the 
pixel on the current depth map with the depth map value of the pixel 
on the hypothetical depth map; and 

a replacement device replacing the depth map value of the pixel 
on the current depth map with the corresponding depth map value of 
the pixel on the hypothetical depth map if the compared depth map 
value of the pixel on the hypothetical depth map has a higher 
probability of accurately representing the reference image than does 
the compared depth map value of the pixel on the current depth map. 

19. The system according to claim 18, wherein the comparison device terminates 
processing once the difference between the depth map values of the current depth map and 
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the depth map values of the derived hypothetical depth map reaches a predetermined 
minimum. 

20. The system according to claim 18, wherein the comparison device compares 
the depth map values a plurality of times across all pixels of the reference image until the 
difference between the depth map values of the current depth map and the depth map values 
of the derived hypothetical depth map reaches a predetermined minimum. 

21. The system according to claim 1 8, wherein the probability of accurately 
representing the reference image is determined according to a Bayesian framework. 

22. The system according to claim 21, wherein the probability of accurately 
representing the reference image is determined according to energy costs and graph cuts. 

23. A system for optimizing an estimate for a depth map of a reference image of 
an object, comprising: 

a first processor estimating a first depth map of a desired view of a reference 
image of an object; and 

a second processor comprising the following for each of a plurality of derived 
hypothetical depth maps of the reference image: 

a third processor applying a Bayesian probability framework to 
determine the optimum depth map value between the two depth maps for every pixel within 
both the first depth map and the derived hypothetical depth map, wherein said determination 
is accomplished by minimizing the energy costs associated with graph cuts between 
neighboring pixel pairs; and 

a replacement device replacing the depth map value in the first 
depth map with the optimum depth map value. 
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